Papers with black-box LLMs
Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has shown that intent detection enhances LLMs’ moderation guardrails, but the robustness of these guardrail mechanisms under malicious manipulations remains under-explored. |
| Approach: | They propose a two-stage intent-based prompt-refinement framework that first transforms harmful inquiries into structured outlines and further reframes them into declarative-style narratives. |
| Outcome: | The proposed framework outperforms several cutting-edge jailbreak methods and evades even advanced Intent Analysis (IA) and Chain-of-Thought (CoT)-based defenses. |
ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddings (2024.emnlp-main)
Copied to clipboard
| Challenge: | Attaching suffixes to harmful instructions can hack the defense of Large language models (LLMs) However, due to the unreadable of adversarial suffix, it can be relatively easily penetrated by common defense methods such as perplexity filters. |
| Approach: | They propose an algorithm to embed adversarial suffixes into coherent and understandable text to attack Large language models (LLMs) using a Advbench dataset. |
| Outcome: | The proposed approach reduces the computation time of adversarial suffixes and achieves a much better attack success rate than existing techniques. |
Modular Pluralism: Pluralistic Alignment via Multi-LLM Collaboration (2024.emnlp-main)
Copied to clipboard
Shangbin Feng, Taylor Sorensen, Yuhan Liu, Jillian Fisher, Chan Young Park, Yejin Choi, Yulia Tsvetkov
| Challenge: | Existing alignment paradigms for large language models learn an averaged human preference and struggle to model diverse preferences across cultures, demographics, and communities. |
| Approach: | They propose a modular framework that "plugs" into a base LLM a pool of smaller but specialized community LMs where models collaborate in distinct modes to support three modes of pluralism: Overton, steerable, and distributional. |
| Outcome: | The proposed framework “plugs into” a base LLM a pool of smaller but specialized community LMs, where models collaborate in distinct modes to support three modes of pluralism: Overton, steerable, and distributional. |
MAQA: Evaluating Uncertainty Quantification in LLMs Regarding Data Uncertainty (2025.findings-naacl)
Copied to clipboard
| Challenge: | despite advances in large language models, they still produce false but incorrect responses. |
| Approach: | They propose a new benchmark for large language models that requires more than two unambiguous answers . they also assess 5 different uncertainty quantification methods in the presence of data uncertainty. |
| Outcome: | The proposed method fails in multi-answer question answering tasks compared to single-answered questions . entropy- and consistency-based methods effectively estimate model uncertainty, the authors show . |
Learning to Instruct: Fine-Tuning a Task-Aware Instruction Optimizer for Black-Box LLMs (2025.findings-emnlp)
Copied to clipboard
Yunzhe Qi, Jinjin Tian, Tianci Liu, Ruirui Li, Tianxin Wei, Hui Liu, Xianfeng Tang, Monica Xiao Cheng, Jingrui He
| Challenge: | Learning to Instruct is a new paradigm for black-box LLMs with inaccessible internal states. |
| Approach: | They propose a new paradigm that formulates instruction optimization as an LLM fine-tuning objective for a white-box “instruction engineer” LLM. |
| Outcome: | The proposed framework outperforms strong baselines in performance and efficiency. |
PIG: Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimization (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to evaluate privacy leakage in LLMs use memorized prefixes or simple instructions to extract data, which well-aligned models can easily block. |
| Approach: | They propose a framework targeting Personally Identifiable Information (PII) that uses in-context learning to build a privacy context and iteratively updates it with three gradient-based strategies to elicit target PII. |
| Outcome: | The proposed framework outperforms baseline methods and achieves state-of-the-art (SoTA) results on four white-box and two black-box LLMs. |
Chain of Attack: Hide Your Intention through Multi-Turn Interrogation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing jailbreak attacks focus on single-turn dialogue scenarios, leaving vulnerabilities in multi-turn contexts inadequately explored. |
| Approach: | They propose an optimal interrogation principle to conceal the jailbreak intent and introduce a multi-turn attack chain generation strategy called CoA. |
| Outcome: | The proposed method shows that black-box LLMs exhibit insufficient resistance under multi-turn interrogation, with more advantages (ASR, 83% vs 64%) |
Crabs: Consuming Resource via Auto-generation for LLM-DoS Attack under Black-box Settings (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on white-box attacks focus on black-box LLMs, leaving black- box scenarios underexplored. |
| Approach: | They propose an automated algorithm designed for black-box LLMs that constructs the DoS Attack Tree and expands the node coverage to achieve effectiveness under black- box conditions. |
| Outcome: | The proposed algorithm can be used to build a DoS Attack Tree and expand the node coverage to achieve effectiveness under black-box conditions. |
Data Contamination Calibration for Black-box LLMs (2024.findings-acl)
Copied to clipboard
| Challenge: | Despite the rapid advancements of Large Language Models, the unchecked ultra-large-scale training sets introduce a series of potential risks like data contamination. |
| Approach: | They propose a method to detect contaminated training data and diminish the contamination effect by using a to-be-released dataset. |
| Outcome: | The proposed method outperforms existing methods by at least 4.5% on more 4 dataset formats, with more than 10 base LLMs. |
Dynamic Retriever for In-Context Knowledge Editing via Policy Optimization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models excel at factual recall yet propagate stale or incorrect knowledge. |
| Approach: | They propose a framework that allows users to rank demonstrations by editing reward . it uses a *learnable threshold* to prune low-value examples, reducing edit success by 17.1% . |
| Outcome: | The proposed framework improves edit success by 17.1% and reduces latency by 41.6% on the CounterFact benchmark. |
ACING: Actor-Critic for Instruction Learning in Black-Box LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have impressive capabilities across tasks like classification, summarization, and reasoning. |
| Approach: | They propose an actor-critic reinforcement learning framework that formulates instruction optimization as a stateless, continuous-action problem. |
| Outcome: | The proposed framework outperforms human-written prompts in 76% of instruction-induction tasks with gains of 33 points and 10-point improvement over baseline. |
ExpNote: Black-box Large Language Models are better Task Solvers with Experience Notebook (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown great power in solving various tasks but fail in many specific tasks. |
| Approach: | They propose a framework to help black-box LLMs better adapt to unfamiliar tasks by reflecting and noting experiences from training data and retrieving them from external memory during testing. |
| Outcome: | The proposed framework improves the performance of black-box Large Language Models on multiple tasks and demonstrates that it is a good choice for the future. |